United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
I nihil Stall-, l'atint and Trademark Office 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



10/74*37 1 



FILING DATE 



I2/2W2007 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



32566 7590 10/08/2008 

PATENT LAW GROUP LLP 
2635 NORTH FIRST STREET 
SUITE 223 

SAN JOSE, CA 95134 



HUBER, JEREMIAH C 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



l/ffflrC? nVrliUli Otfff Iff ids y 


Application No. 

10/748,371 


Applicant(s) 

ZHEN ET AL. 


Examiner 

JEREMAIAH C. HUBER 


Art Unit 

2621 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 15 August 2008 . 
2a )□ This action is FINAL. 2b)^ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1.4.5.7 and 9-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 1.4.5.7 and 9-20 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 29 December 2003 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1) ^| Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26 d (Rev e 08-06r 



Office Action Summary 



Part of Paper No./Mail Date 20080930 



Application/Control Number: 10/748,371 Page 2 

Art Unit: 2621 

DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
8/15/2008 has been entered. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1, 7, 9, 10 , 12, 13, 15, 16, 18 and 20 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Pyramidal Implementation of the Lucas Kanade Feature 
Tracker Description of the Algorithm (hereafter Bouguet) in view of Clayton 
(20040017507) and in further view of Motion Compensated Image Interpolation 
(hereafter Cafforio) and in further view of Aoki (5982440). 

In regard to claim 1 Bouguet discloses a method including: 
generating first and second image pyramids comprised of a plurality of corresponding 
image levels of gradual resolution (Bouguet 2.1 note pg. 2 paragraph 3 note pyramidal 
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representations of images I and J are composed of gradual resolutions e.g. 640x480, 
320x24, 160x120 etc.); 

warping a first image level of a first image pyramid with a motion field (Bouguet 
2.1 for image pyramid and 2.3 particularly pg. 5 note B k new translated image); 
determining a residual motion field from the warped first level image of the first image 
pyramid and a corresponding first level image of the second image pyramid (Bouguet 
2.3 particularly pg. 5 note residual v k ) ; 

when an error value is not less than a threshold, adding the residual motion field 
to the motion field and repeating the above steps (Bouguet 2.3 particularly pg. 5 note 
eq. 31 adding residual motion v k further computation continues until pixel is smaller than 
a threshold); and 

when an error value less than the threshold performing the above steps using the 
second level image of the first and second image pyramids (Bouguet 2.2 particularly pg. 
2 note results are propagated from deeper to upper levels). 

It is noted that Bouguet discloses comparing a pixel error to a threshold rather 
than a residual motion vector. However, Clayton discloses a motion compensation 
method wherein a residual motion is compared to a threshold as a test to stop an 
iterative process (Clayton par. 130 note iteration causes a change in motion vector to be 
less than a threshold). It is therefore considered obvious that one of ordinary skill in the 
art would recognize the advantage of include a residual motion vector threshold as 
taught by Clayton in the invention of Bouguet in order to reduce computation time 
incurred by computing pixel errors. 
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It is further noted that neither Bouguet nor Clayton disclose generation of an 
intermediate image on a pixel by pixel basis. However, Cafforio discloses a method for 
motion compensated interpolation in which an intermediate image is generated between 
first and second images on a pixel by pixel bases using a motion field (Cafforio Section 
IV note part A motion field estimated on a pixel by pixel basis and part B for two 
methods of generating an intermediate image also note Figs. 8-9). It is therefore 
considered obvious that one of ordinary skill in the art at the time of the invention would 
recognize the advantage of including intermediate image generation as taught by 
Cafforio in the invention of Bouguet in view of Clayton in order to interpolate missing 
images. 

It is further noted that neither Bouget, Clayton, nor Cafforio disclose inserting 
generated intermediate images between original video images in order to increase the 
total number of images and produce a slow motion effect. However, at the time of the 
invention it was common and notoriously well known to insert generated intermediate 
images between original video images in order to produce a slow motion effect by 
increasing the number of frames as disclosed by Aoki (Aoki Figs. 3 and 4 and col. 4 
lines 21-64 note Fig. 3 interpolated images are inserted between reproduced pictures to 
increase the total number of frames). It is therefore considered obvious that one of 
ordinary skill in the art would recognize the advantage of applying the invention of 
Bouget in view of Clayton and in further view of Cafforio to the application of slow 
motion video by as taught by Aoki in order to produce video with a smooth slow motion 
effect as suggested by Aoki (Aoki col. 4 lines 61-64). 
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In regard to claim 7 refer to the statements made in the rejection of claim 1 
above. Cafforio further discloses selecting first and second pixels paired by a motion 
vector and determining a position of a pixel in the intermediate image on a recurring 
pixel by pixel basis (Cafforio sect. IV B note first method in first two paragraphs motion 
field is used to find corresponding pixels on a pixel by pixel basis and determine the 
position of the pixel in the intermediate frames using a fraction of displacement). It is 
noted that Cafforio does not explicitly state a value that is determined for the 
interpolated pixel. However, it is inherent to the function of generating frames for a value 
to be assigned to every frame pixel. Further, Cafforio implies that the value given to the 
interpolated pixel is the value of the first or second pixel (Cafforio Sect IV B paragraph 1 
note position of the corresponding points in the missing frame). 

In regard to claim 9 refer to the statements made in the rejection of claim 1 
above. Bouguet further discloses: 

providing a first second image pyramids associated with first and second frames 
comprised of a plurality of corresponding image levels of gradual resolution (Bouguet 
2.1 note pg. 2 paragraph 3 note pyramidal representations of images I and J are 
composed of gradual resolutions e.g. 640x480, 320x24, 160x120 etc.); 
computing a motion field comprising a plurality of motion vectors that respectively 
associate each pixel in the first image with a corresponding pixel in the second image, 
wherein the motion field is computed recursively through the image levels from a 
coarsest resolution to the finest (Bouguet 2.2 and 2.3 note 2.2 on pg. 2 feature tracking 
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establishes a displacement vector for each pixel, also note optical flow is computed 
from the deepest or coarsest level to the upper or finest levels) 

In regard to claim 10 refer to the statements made in the rejection of claims 1 and 
9 above. Bouguet further discloses performing a computation loop to derive a motion 
field based on for a current level based on a motion field of a coarser level wherein the 
previous motion field is initially assigned to the current motion field (Bouguet 2.2 and 2.3 
note pg. 2 results are propagated to a next lower level as an initial guess). 

In regard to claim 12 refer to the statements made in the rejection of claim 10 
above. Bouguet further discloses that the motion field is obtained by applying the 
computation loop to the finest level (Bouguet 2.2 pg. 2 note optical flow is refined up to 
the level of the original image). 

In regard to claims 13, 15, 16, 18 and 20 refer to the statements made in the 
rejection of claims 1, 7, 9, 10 and 12 above. 

Claims 4, 14 and 17 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bouguet in view of Clayton and further in view of Cafforio and The Laplacian Pyramid as 
a Compact Image Code (hereafter Burt) and in further view of Aoki. 
In regard to claims 4, 14 and 17 refer to the statements made in the rejection of claims 
1 , 9 and 16 above. Bouguet further discloses filtering of images during generation 
(Bouguet 2.1 pg. 2). It is noted that neither Bouguet, Clayton nor Cafforio disclose 
generating Laplacian pyramids. However, at the time of the invention the use of 
Laplacian image pyramids was common and notoriously well known in the art as is 
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evidenced by Burt (Burt note Laplacian Pyramid pgs. 535-537). It is therefore 
considered obvious that one of ordinary skill in the art at the time of the invention would 
recognize the advantage of utilizing Laplacian Pyramids in the invention of Bouguet in 
view of Clayton and in further view of Cafforio and in further view of Aoki as was well 
known in the art in order to enhance image features as suggested by Burt (Burt pg. 535 
col. 2). 

Claims 5, 11 and 19 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bouguet in view of Clayton and further in view of Cafforio and Determining Optical Flow 
(hereafter Horn). 

In regard to claims 5, 1 1 and 19 refer to the statements made in the rejection of claims 
1,10 and 18 above. It is noted that neither Bouguet, Clayton nor Cafforio disclose use 
of a Horn and Schunk motion estimation algorithm. However, at the time of the invention 
use of the Horn and Schunk motion algorithm was common and notoriously well known 
in the art as is evidence by Horn (Horn pgs. 1-14 describe the Horn and Schunk 
algorithm). It is therefore considered obvious that one of ordinary skill in the art at the 
time of the invention would recognize the advantage of utilizing the Horn and Schunk 
algorithm in the invention of Bouguet in view of Clayton and in further view of Cafforio 
and in further view of Aoki in order to have a robust estimation of motion as suggested 
by Horn (Horn Abstract). 
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Response to Arguments 

Applicant's arguments with respect to claims 1 , 4, 5, 7 and 9-20 have been 
considered but are moot in view of the new ground(s) of rejection. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JEREMAIAH C. HUBER whose telephone number is 
(571 )272-5248. The examiner can normally be reached on Mon-Fri 8:00 a.m. - 4:30 
p.m.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571)272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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